CLAIMS 

1. y\ facial expression transformation method comprising: 

defining a code book containing data defining a first set of facial 
expressions of a first person; 

providin&data defining a second set of facial expressions, the second set of 
facial expressions\providing a training set of expressions of a second person who 
is different from tha first person; 

deriving a transformation function from the training set of expressions and 
corresponding expressions from the first set of expressions; and 

applying the transformation function to the first set of expressions to 
provide a synthetic set onexpressions. 

2. The method of claim 1, wherein the training set of expressions 
contains fewer expressions than the code book. 

3. The method of \claim 1 , wherein the transformation function 
compensates for differences in \the size and shape of the faces of the first and 
second persons. \ 

4. The method of claim K wherein said deriving of the transformation 
function comprises computing a linear transformation from one set of expressions 
to another. * 
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5. The\ method of claim 1, wherein the deriving of the transformation 
function comprises: 

representing\ each expression as a 3m- vector that contains x, y, z 
displacements at m standard sample positions; and 

computing a set of linear predictors aj 9 one for each coordinate of g a , given 
a set of n expression vectors for a face to be transformed, g a i... n , and a 
corresponding set of vectors for a target face, gti... n9 by solving 3m linear least 
squares systems of the following form: 

\ W ' gat = gb\j] 9 i = l...w 

6. The method of claim 5, wherein said computing comprises using only 
a subset of points for^ach g aJ . 

7. The method <af claim 6, wherein said using comprises using only 
points that share edges with aSstapoard sample point under consideration. 

8. The method of dlaiin A further comprising controlling the spread of 
singular values when competing a paAidoinverse to solve for the a y . 

9. The method of claim 8, wherein said controlling the spread comprises 
zeroing out all singular values less than aau, where a } is the largest singular value 
of the matrix. \ 
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10. The methou of claim 1, wherein said providing data defining a 
second set of facial expressions comprises: 

illuminating the secqnd person's face with illumination; and 
contemporaneously capturing structure data describing the face's structure 

and reflectance data describing reflectance properties of the face from the 

illumination. 

11. Theunethod of claim 10, wherein said illuminating comprises: 
using multiple light sources, one of which projecting a pattern on the 

second person's face rrom which the structure data can be ascertained; 

at least one of tlAlight sources comprising an infrared light source; 
at least one of the light sources being polarized; and 

said capturing comprising using a camera having a polarizer that suppresses 
specularly-reflected light so that diffuse component reflection data is captured. 

12. The method of clJiim l\ wherein said providing data defining a 
second set of facial expressions <romprises: 

illuminating the second person's face with a first polarized light source that 
is selected so that specularly-suppressed reflective properties of the face can be 
ascertained; \ 

illuminating the second person's face with a second structured light source 
that projects a pattern onto the face, whiles simultaneously illuminating the face 
with the first polarized light source; and \ 

capturing both specularly-suppressed reflection data and structure data from 
the simultaneous illumination. \ 
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13. Yrhe method of claim 12, wherein the light sources provide light at 
different frequencies. 



14. TheYmethod of claim 12, wherein the light sources provide infrared 



light. 



15. The method of claim 12, further comprising processing the captured 
data to provide both (a) uata that describes dimensional aspects of the face and (b) 
data that describes diffusareflective properties of the face. 



16. The method 
second set of facial express! 
illuminating the second 



mim\l/wherein said providing data defining a 
Ises: 

srson'svface with multiple different light sources; 



measuring range map datagram said illuminating; 
measuring image data from Wd illuminating; 
deriving a 3 -dimensional surmce from the range map data; 
computing surface normals to tlae 3-dimensional surface; and 
processing the surface normals and the image data to derive an albedo map. 



17. The method of claim 16, wherein at least one of the light sources is 
polarized. 
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18. 

polarized. 



19. One or more computer-readable media having computer-readable 
instructions thereon which, when executed by a computer, cause the computer to: 

operate on Wining set of expressions from one person and corresponding 
expressions from £^ code book of another person to compute a linear 
transformation functiomfrom the training set and their corresponding expressions; 
and \ 

apply the transformation function to a plurality of expressions from the 
code book to provide a syntheru: set of expressions. 

20. The computer-readaftde media of claim 19, wherein the instructions 
cause the computer to use the synthetic set of expressions to transform expressions 
from the one person into expressions orthe other person. 

21. The computer-readable mediavof claim 20, wherein the instructions 
cause the computer to transform expressions from the one person that are different 
from those expressions comprising the code boolk expressions. 

22. The computer-readable media of claiiA 20, wherein the instructions 
cause the computer to transform expressions by transmitting at least one index of a 
synthetic expression to a receiver that can reconstruct the expression. 
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^3. The computer-readable media of claim 20, wherein the instructions 
cause the^computer to transform facial expressions. 

24. \ A facial expression transformation system comprising: 

a code iDook embodied on a computer-readable medium, the code book 
containing data defining a first set of facial expressions of a first person; 

data embodied on a computer-readable medium, the data defining a second 
set of facial expressions, the second set of facial expressions providing a training 
set of expressions of a second person who is different from the first person; and 

a transformation processor configured to derive a transformation function 
from the training set of expressions and corresponding expressions from the first 
set of expressions. \ 

25. The expression transformation system of claim 24, wherein the 
transformation processor comprises a^iinear transformation processor. 

26. The expression transformation system of claim 24 further 
comprising a synthetic set of expressions embodied on a computer-readable 
medium, the synthetic set of expressions being derived by applying the 
transformation function to the code book expressions. 

27. The expression transformation system of claim 24, wherein the 
transformation function compensates for differences i\ the size and shape of the 
faces of the first and second persons. \ 
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28. Vhe expression transformation system of claim 24p wherein the 
transformation ptocessor derives the transformation function by: 

representing each expression as a 3m- vector that contains x, y, z 
displacements at m srandard sample positions; and 

computing a se\of linear predictors a j9 one for each coordinate of g a , given 
a set of n expression Yvectors for a face to be transformed, g a i... n > an d a 
corresponding set of vectors for a target face, gtj... n , by solving 3m linear least 
squares systems of the followng form: 

<*j f *Y« = £*'[/], i = l-w 

29, A facial expression Wnsformation system comprising: 
a transmitter comprising: \ 

a facial illumination system that is configured to provide multiple 
different light sources at the same time for illuminating a subject's face; 

a data-capturing system configured to capture both structure data and 
reflectance data from the subject^ face when illuminated by the facial 
illumination system; and \ 
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a first code book of synthetic expressions that have been synthesized 
by: \ 

\receiving a training set of expressions provided by the 
subject;\ 

computing a transformation function using the training set of 
expressions^ and corresponding unsynthesized code book 
expressions; and 

applyingythe transformation function to all of the expressions 
in the code book;\and 
a receiver communicatively linked with the transmitter and comprising: 
a reconstruction module for reconstructing facial images; and 
a second code book containing the same synthetic expressions as the 
first code book; and \ 

the transmitter being configured to: 

capture additional expressions of the subject; 

search the first code pook for a corresponding or near 
matching expression; and \ 

transmit an index of a corresponding or matching code book 
expression to the receiver for faciM image reconstruction by the 
reconstruction module. \ 

30. The expression transformation system of claim 29, wherein the 
illumination system comprises at least one polarized light source. 
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31. Uhe expression transformation system of claim 29, wherein the 
illumination system comprises multiple polarized light sources. 

32. The Wpression transformation system of claim 29, wherein the 
illumination system\ comprises a patterned light source configured to project a 
pattern onto the subject's face. 
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33. The expression transformation system of claim 29, wherein the 
illumination system comprises an infrared patterned light source configured to 
project a pattern onto the subject's face. 

34. The expression transformation system of claim 29, wherein the 
different light sources are all infrared light sources. 

35. A method of animating facial features comprising: 

defining a subdivision surface that approximates geometry of a plurality of 
different faces; and 

fitting the same subdivision sikface to each of the plurality of faces. 

36. The method of claim 35a wherein said defining comprises defining 
the subdivision surface with a coarse meih structure. 

37. The method of claim 36, \ wherein the coarse mesh structure 
comprises a triangular mesh. 
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38. Thfe method of claim 35 5 wherein said fitting comprises performing 
a continuous oprimization operation over vertex positions of the subdivision 
surface. \ 

39. The method of claim 35, wherein said fitting comprises fitting the 
subdivision surface tovthe faces without altering the connectivity of a mesh that 
defines the subdivision aurface. 

40. The methodW claim 35, wherein said fitting comprises minimizing 
a smoothing functional associated with a mesh that defines the subdivision 
surface. \ 

41. The method of 4laim 35, wherein said fitting comprises selecting 
one or more constraints associated with a mesh that defines the subdivision surface 
and fitting those constraints directW to corresponding points on the faces. 

42. The method of claim 4tt, wherein the constraints are associated with 
one of the eyes, nose and mouth. \ 

43. The method of claim 35, therein said fitting comprises minimizing 
a functional that includes terms for distance, smoothness, and constraints. 

44. The method of claim 35, wherein said fitting comprises solving a 
sequence of linear least-squares problems. \ 
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45. \ One or more computer-readable media having computer-readable 
instructions thereon which, when executed by one or more computers, cause the 
one or more computers to implement the method of claim 35. 

46. A method of animating facial features comprising: 

defining a subdivision surface that approximates geometry of a plurality of 
different faces; \ 

fitting the same subdivision surface to each of the plurality of faces to 
establish a correspondence aetween the faces; and 

using the correspondence between the faces to transform an expression of 
one face into an expression of another face. 

47. A method of animating facial features comprising: 

measuring 3 -dimensional dataVor a plurality of different faces to provide 
corresponding face models; \ 

defining only one generic face mbdel that is to be used to map to each 
corresponding face model; \ 

selecting a plurality of points on the\generic face model that are to be 
mapped directly to corresponding points oV each of the corresponding face 
models; and \ 

fitting the generic face model to each of the corresponding face models, 
said fitting comprising mapping each of the selected points directly to the 
corresponding points on each of the corresponding race models. 
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48. The method of claim 47, wherein: 

said defining comprises defining a subdivision surface from a base mesh 
structure, the subdivision surface containing a plurality of vertices and 
approximating the geometry of the face models; and 

said fitting comprises manipulating only the positions of the vertices of the 
subdivision surface. \ 

49. The methou of claim 47, wherein said fitting comprises 
manipulating a base mesh tliat defines a subdivision surface. 

50. The method of claim 47, wherein said fitting comprises 
manipulating a base mesh that (defines a subdivision surface without altering the 
connectivity of the base mesh. \ 

51. The method of claim 47, wherein said measuring comprises using a 
laser range scan to measure the 3-dimensional data. 
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